Camera-based retail scanning systems commonly capture between 30 and 100 frames of imagery every second. If real-time operation is to be achieved, such systems can only allocate a short interval to process each image. For example, if a system captures 60 frames each second, it can devote a maximum of 16.6 milliseconds to each image.
This time constraint is a particular challenge because product-identifying indicia (e.g., a barcode or watermark data) may appear anywhere in the captured frames. Moreover, each captured image is commonly a synthesis of several different images—captured from different viewpoints by an optical arrangement including multiple mirrors. Searching the full extent of such a composite image in search of a decodable indicia is typically too time consuming. It is generally preferable to examine a limited number of image excerpts.
Yet another complication is that the pose at which any indicia is captured is unknown. The indicia may be unrecognizable due to the perspective distortion introduced by the pose. A variety of brute-force compensations can be applied to try and counter-act possible distortions to improve detection, but only a limited number of such compensations can be tried in the limited period available.
If nine image excerpts are examined from each image, and 50 different perspective corrections are trialed for each, then there are 450 different perspectively-transformed image excerpts to prepare, and attempt to decode, every 16 milliseconds. This is beyond the capabilities of the processing hardware commonly provided in retail scanners, and associated point of sale terminals.
In accordance with one aspect of the present technology, an ordered listing of processing parameters is prepared. Each entry includes a particular location from which an image excerpt may be taken, and a particular set of geometrical correction factors that should applied to that excerpt prior to decoding. A processor receives an input image and tries decoding the first-specified excerpt after applying the first-specified correction factors. If time remains, the processor tries decoding the second-specified excerpt after applying the second-specified correction factors. The process continues down the ordered listing, with the processor making repeated attempts to decode indicia from the input image until the available time runs out. At that point, any decoded indicia data is output by the processor to an associated point of sale terminal, and the process begins anew with the next frame of captured imagery.
The ordered listing of processing parameters can be derived from reference scanner images that are analyzed exhaustively. In an illustrative embodiment, imagery captured during normal operation of the scanner by a checkout clerk is processed. A thousand images may be used—selected randomly from those depicting product (i.e., that are not simply blank). For each such reference image, each of various image excerpt locations is trialed with each combination of perspective correction factors. (The perspective correction factors can be at discrete values, such as 0, 10, 20, 30, 40, and 50 degrees of tilt correction, and 0, 45, 90, 135, 180, 225, 270, and 315 degrees of bearing correction). Each set of processing parameters will lead to successful decoding of indicia from some number of the reference images (or, sometimes, none).
A set cover strategy is desirably used. In particular, the set of processing parameters that led to successful decoding of the largest number of reference images is placed at a first position in the ordered list. Those successfully-decoded reference images are then removed from consideration, and the decoding results are examined to identify the set of processing parameters that led to decoding of the largest number of the images that remain in the reference set. This set of processing parameters is placed at the second position in the ordered list. Again, the reference images that were successfully decoded using this second set of processing parameters are removed from the reference set. The decoding results are then examined to identify the set of processing parameters that led to successful decoding of the largest number of frames in the remaining reference set. These parameters are entered at the third position in the listing, and the set of reference images is reduced accordingly. This process can continue until each set of processing parameters is assigned a position in the ranked listing.
It will be recognized that this method allows behavior of a scanning system to be optimized to particular operators, who may have different habits of presenting products to the cameras. These different habits will cause different sets of processing parameters to be preferred in decoding. When a new checker signs-on to a point of sale terminal, the ranked listing of processing parameters for that checker is loaded into memory and used for that checker's shift. The same approach can be used for individual shoppers, who are identified at the start of a self-checkout transaction, e.g., by a store loyalty card.
The ordering of processing parameters in the list can evolve. The system can compile a set of reference images during normal operations (e.g., at moments when processing resources are otherwise idle, such as when waiting for customer payment action), to determine which order of parameter application yields the best frame-decoding coverage, currently. If bottles of soda are on sale for a quarter each, and their number spikes at the checkout registers, then the parameter listing can adapt to rank more highly those parameters that seem to work best with a soda bottle-rich mix of products. If a checker reports to work one day with an adhesive bandage on a finger, and that bandage causes the checker to change the way they handle products (e.g., gripping cans by their sides instead of their tops), then the order in which decoding parameters are tried can adapt to reflect the changed statistical breakdown of product poses.
The foregoing and other features and advantages of the present technology will be more readily apparent from the following detailed description, which proceeds with reference to the accompanying drawings.